Coordinated Workload Scheduling: An Emerging Application Domain for Mechanism Design

The process of distributing and coordinating parallel workloads across many computation or storage nodes is a hard scheduling optimization problem. Furthermore, in distributed systems where nodes require monetary incentive-based payment schemes to schedule workloads efficiently and fairly, or a central coordinator might not have enough information to make the best system-wide decisions, framing scheduling problems in the context of a monetary incentive-based mechanism design can lead to globally efficient and fair solutions while nodes make individually-optimizing decisions. We motivate the application of mechanism design in two areas of coordinated workload scheduling. We first apply an auction that uses a Vickrey-Clarke-Groves mechanism to improve the scheduling process in a free distribution setting, where many nodes may service any workload. We then study an environment with a fixed distribution setting, where a fixed set of nodes is assigned to each workload, and show that the problem of coordinating even a single timeslice of these workloads across all workloads and nodes in the system is hard. While the solution to workload coordination in this fixed distribution setting is still an open research problem, we frame the problem in the context of a mechanism design by computing a number of derived properties of schedule allocations, showing how these properties may be used in a payment scheme that gives the nodes incentives for fair and efficient schedules, and define the goals of a mechanism that distributes the assessment of possible schedule allocations over many nodes in the system.